Qualitative similarities and differences in visual object representations between brains and deep networks

Abstract Deep neural networks have revolutionized computer vision, and their object representations across layers match coarsely with visual cortical areas in the brain. However, whether these exhibit qualitative patterns seen human perception or brain remains unresolved. Here, we recast well-known perceptual phenomena terms of distance comparisons, ask they are present feedforward deep trained for recognition. Some were randomly initialized networks, such as global advantage effect, sparseness, relative size. Many others after recognition training, Thatcher mirror confusion, Weber’s law, size, multiple normalization correlated sparseness. Yet other absent 3D shape processing, surface invariance, occlusion, natural parts advantage. These findings indicate sufficient conditions emergence brains offer clues to properties that could be incorporated improve networks.